Linear-Time Inverse Covariance Matrix Estimation in Gaussian Processes

The computational cost of Gaussian process regression grows cubically with respect to the number of variables due to the inversion of the covariance matrix, which is impractical for data sets with more than a few thousand nodes. Furthermore, Gaussian processes lack the ability to represent conditional independence assertions between variables. We describe iterative proportional scaling for directly estimating the precision matrix without inverting the covariance matrix, given an undirected graph and a covariance function or data. We introduce a variant of the Shafer-Shenoy algorithm combined with IPS that runs in O(nC3)-time, where C is the largest clique size in the induced junction tree. We present results on synthetic data and temperature prediction in a real sensor network.